Eye gaze responsive virtual reality headset

ABSTRACT

A headset for providing images to a user is provided. The headset is configured to attach to a user&#39;s head and includes displays positioned adjacent to a user&#39;s eyes. A display support arm is connected to the displays. A motor is connected to the display support arm and drives the display support arm between multiple positions. A camera is positioned adjacent to the user&#39;s eyes and records the user&#39;s eyes. A processor receives data from the camera regarding a position and movement of the user&#39;s eyes. The processor uses the data from the camera to drive the motor and to position the display support arm with respect to the user&#39;s eyes such that the displays maintain a position adjacent to the user&#39;s eyes.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. provisional patentapplication 62/236,964, filed on Oct. 4, 2015, the entire contents ofwhich is incorporated by reference as if fully set forth herein.

FIELD OF THE INVENTION

The present invention relates to a head-mounted display device and acontrol method for the head-mounted display device.

BACKGROUND

The present state of the art includes a head-mounted display (“HMD”)device that is mounted on the user's head and includes one or moredisplay devices for presenting stereoscopic images to a user's left eyeand a user's right eye so that the user perceives images in threedimensions. The images in the HMD can be related to a virtual or realworld scene and the images are related to the user's head's orientationso that the user feels immersed in a scene presented by the display. Thedisplay devices in the HMD device may be liquid crystal displays and alight source that guides the generated image light to the eyes through aprojection optical system and a light guide plate for proper vision atsmall distance.

In the HMD of the present state of the art, the resolution of the imagespresented to the eye is limited by the capacity of the displays. Thehigher resolution of a display presents an improved image in terms ofclarity and definition. The angle of view of systems of the presentstate of the art is determined by the size of the displays in front ofthe eyes. If the display in front of the eyes is increased, then theangle of view is also increased. Systems of the present state of the artdo not allow the user to move their eyes left and right outside theviewing cone, which is determined by the angle of view of the displays.Instead, the user changes his or her perspective in the virtual or realworld by rotating his head while keeping his eyes generally straightahead focused on the displays. The need to move one's head for moderatechanges in perspective may place an unnecessary burden on the user'sneck and shoulder muscles. In addition to this ergonomic problem, auser's natural tendency is to change their view by slightly adjustingthe direction of his eyes. Therefore, the method of changing thedirection of view by changing the angle of the entire head as isrequired with current headsets is not natural and can be strenuous onthe user.

It would be desirable to provide a virtual reality headset that allows auser to explore a corresponding virtual world by allowing minor eyemovement by the user to adjust the corresponding view in the virtualworld.

SUMMARY OF THE INVENTION

The present invention provides, generally, a wearable device for viewingimages, having displays that move in response to the movement of auser's (wearer's) eye movement. The user's eye movement is detected,sensed, measured, or otherwise captured, and processed. The results ofsuch processing are used to move the displays to correspond to the eyemovement.

A wearable device for a user's head is disclosed that presents images toa user's left eye and a user's right eye. The images presented to theuser are related to a virtual world, an animation, digital images, or torecorded images of a real-world or other recording, and the images arestereoscopic so that world is perceived by the user as beingthree-dimensional. In one embodiment, the images are presented to theleft eye and the right eye through a left display and a right display,with each display being adjustable and movable, either together orindependently, and the displays are oriented in the direction of theleft eye and the right eye as the eyes move.

Generally, the displays move and orient in response to eye movement. Thesystem records or otherwise detects the movement the user's eyes thoughone or more cameras or other sensors focused on the eyes, and uses theimagery captured by the camera to calculate data related to the relativeposition of the user's eyes. The displays are attached to a support thatis moved via motors that are controlled by a controller. The datarelated to movement of the user's eyes is used to position the displays.The data related to movement of the user's eyes is also used to controlthe image displayed on the displays. Therefore, the effective field ofview of the headset is increased without deterioration of resolution ofthe image presented to the user. This allows the user to more naturallyand comfortably view the virtual world. A user can gaze around a virtualworld using eye movement, as in the real world.

In one embodiment, a user's gaze is detected through one or morecameras, or other sensors or detectors, viewing the user's eyes andthrough software that detects the movement of parts of the eyes, e.g.the pupils of the eyes. The system of the present invention includesdisplays that are in the field of view of the user's eyes. The displaysare moved and repositioned as the user's eyes move so that the displaysare constantly positioned in front of the user's eyes. The system of thepresent invention also causes the image displayed to the user to changedepending on the movement of the user's eyes so that the displayed imageis related to the field of view commensurate with the state andpositioning of the user's eyes.

DESCRIPTION OF THE DRAWINGS

The foregoing Summary and the following detailed description will bebetter understood when read in conjunction with the appended drawings,which illustrate a preferred embodiment of the invention. In thedrawings:

FIGS. 1A and 1B illustrate a first embodiment of a headset according inan upward and downward position.

FIG. 2 illustrates the first embodiment of the headset.

FIG. 3 is a flow diagram illustrating a process for positioning displaysbased on tracking of the user's eyes.

FIG. 4 illustrates an alternate embodiment of the headset includinglenses.

FIGS. 5A and 5B illustrate displays showing an image in two differentorientations based on a user's eyes.

FIG. 6 is a schematic drawing of the headset of FIGS. 1A, 1B, and 2.

FIG. 7 is a schematic drawing of a headset according to anotherembodiment.

DETAILED DESCRIPTION

Certain terminology is used in the following description for convenienceonly and is not limiting. The words “front,” “rear,” “upper” and “lower”designate directions in the drawings to which reference is made. Thewords “inwardly” and “outwardly” refer to directions toward and awayfrom the parts referenced in the drawings. “Axially” refers to adirection along the axis of a shaft. A reference to a list of items thatare cited as “at least one of a, b, or c” (where a, b, and c representthe items being listed) means any single one of the items a, b, or c, orcombinations thereof. The terminology includes the words specificallynoted above, derivatives thereof and words of similar import.

As shown in FIGS. 1A, 1B, and 2, a system 100 is provided for displayingimages to a user. The system 100 includes a headset 11 that can befastened to a user's head or otherwise attached to a user. For example,a strap, multiple straps, or a helmet arrangement can be used to attachthe headset 11 to the user. The system 100 includes two displays 1, 2that are positioned in front of a user's eyes 9, 10. The images providedby the displays 1, 2 can include any type of image, such as a videogame, recorded images, live images, film, three-dimensional virtualworld, or other type of image.

The displays 1, 2 are attached to a display support arm 3. The displaysupport arm 3 is illustrated as forked-type bar in the drawings. Asshown in FIGS. 1A, and 1B, a crossbar 3 a is provided between separateprongs of the display support bar 3. In FIG. 2, the display support arm3 lacks any such crossbar 3 a. One of ordinary skill in the art willrecognize from the present disclosure that alternative configurations ofthe display support arm 3 can be used. For example, the display supportarm 3 could include multiple articulated joints and a plurality of arms.In one embodiment, the displays 1, 2 are each attached to a separatelyarticulated display support arm, and the displays 1, 2, areindependently positioned from each other.

The display support arm 3 is attached to a pitch rotating motor 4, andthe display support arm 3 is configured to be manipulated by the pitchrotating motor 4. The pitch rotating motor 4 moves the display supportarm 3 in three dimensions to move the displays 1, 2 with respect to theuser's eyes 9, 10. The pitch rotating motor 4 is also attached to a yawrotating motor 5 so that the pitch motor 4 and the yaw rotating motor 5control the position of the displays 1, 2 in front of a user's eyes 9,10. Other types of motors can be used, as long as the motors havesufficient a sufficient degree of motion to move the display support arm3. Actuation of the pitch rotating motor 4 and the yaw rotating motor 5is controlled by a processor 7 within the headset 11. The processor 7sends signals to the motors 4, 5 to drive the motors 4, 5 and move thedisplay support arm 3 in multiple directions.

The system 100 also includes a camera 6 for tracking the user's eyes 9,10. The camera 6 is fixed within the headset 11 relatively close to theuser's eyes 9, 10 such that the camera 6 has an unobstructed line ofsight to the user's eyes 9, 10. Data from the camera 6 is transmitted tothe processor 7. The camera 6 preferably includes a pupil tracking lensthat specifically locates and maps movement of the eye. The camera 6 cantrack rotation of the eyes, position of the pupil, or othercharacteristics of the eye. The camera 6 has a very high sampling rate,preferably of at least 30 Hz, more preferably between 50-350 Hz, andmost preferably of at least 1000 Hz. Although only one camera 6 is shownin the figures, one of ordinary skill in the art would recognize thatmultiple cameras 6 can be used.

In one embodiment, the camera 6 is an active light projecting infraredcamera system focused on the user's eyes 9, 10. The camera 6 acquiresdata comprising an image sequence depicting the pupil of the eye in highcontract. Sequential images are analyzed through software in theprocessor 7 to determine a movement vector that maps a deviation of acenter of the pupil from a reference point. This reference point may bethe center of the image or it may be a point detected through acalibration procedure related to an orthogonal, straight forward, gazeof the user with a reference point in the image. In one embodiment, theuser may be prompted to look straight ahead and press a button, i.e. akey on a keyboard, to relate in software running in processor 7 thecurrent image acquired with the orthogonal gaze. The movement vectorthen is used to provide motion input commands to the motors 4, 5, and/orto adjust the images provided to the displays 1,2.

Data from the camera 6 is processed by the processor 7 and is used todrive the motors 4, 5 so that the displays 1, 2 remain in front of theuser's eyes 9, 10 as the user moves their eyes 9, 10. For example, if auser moves their eyes 9, 10 quickly to the left, right, up, or down,then the camera 6 sends data related to the eye movement to theprocessor 7, which then quickly processes this data to provideinstructions to the motors 4, 5 to move the display arms 3. It isappreciated that other types of sensors or detectors may be used tomeasure, track or otherwise determine the position of a user's eyes,such as IR (infra-red) sensors, optical sensors, beam sensors, or lasersensors.

In one embodiment, the headset 11 also includes an accelerometer 12 orother sensor that detects and generates data related the orientation ofthe headset 11. The accelerometer 12 can detect motion, orientation,and/or positioning of the headset 11. Data from the accelerometer 12 istransmitted to the processor 7, and the processor 7 transmits both (1)the data related to the movement of the user's eyes 9, 10, and (2) thedata related to the orientation of the headset 11 to a central processor14 through transmitter/receiver units 8, 13. The processor 7 alsoreceives image data related to a virtual or real world from the centralprocessor 14. The transmitter/receiver unit 13 and the central processor14 can be in the form of a central processing unit (CPU).

FIGS. 1A and 1B show the displays 1, 2 are moved from (1) a firstposition in FIG. 1A when the user's eyes 9, 10 are moved upward to (2) asecond position shown in FIG. 1B when the user's eyes 9, 10 are moveddownward. The camera 6 sends input related to data gathered duringmovement of the user's eyes 9, 10 to the processor 7, and the processor7 then uses this data to provide input to the motors 4, 5. The motors 4,5 then drive the display support arm 3 to different positions based onthe data generated by the camera 6. The processor 7 also provides imagesto the displays 1, 2 based on the position of the user's eyes 9, 10. Forexample, if the user is looking downward, the processor 7 sends imagesto the displays 1, 2 for a downward view. While FIGS. 1a and 1b show auser moving their eyes up and down, one of ordinary skill in the artwill recognize from the present disclosure that other movement of theeyes are captured by the system 100, including a full range of motion,to provide responsive and appropriate images to the displays 1, 2.

FIG. 5A illustrates the displays 1, 2 showing an image in a firstorientation, and FIG. 5B illustrates the displays 1, 2 showing the imagein a second orientation. As shown in FIG. 5A, the displays 1, 2 show animage of two characters in a first orientation in which the entirefigures are visible. FIG. 5A corresponds to a first condition in which auser is looking straight forward. FIG. 5B corresponds to a secondcondition in which a user moves their eyes downward compared to theposition depicted in FIG. 5A. In FIG. 5B, only the lower halves of thefigures are visible. FIGS. 5A and 5B illustrate the concept that as theuser moves their eyes, the image is adjusted to correspond to theposition of the user's eyes.

A flow diagram of a method 300 for controlling the system 100 isprovided in FIG. 3. As shown in FIG. 3, input from the accelerometer 310and input from the camera 320 are used in data processing 330 to controlthe display support arms 340. The display support arm moves the displaysin front of a user's eyes based on movement and orientation of theuser's eyes. Input from the accelerometer 310 and input from the camera320 are also used by the controller in data processing 330 to generateimage data 350 for the displays related to the orientation of headsetand related to the orientation of the user's eyes, and the displays showimages 360 based on the data. The flow diagram of the method 300generally discloses two input steps, both the accelerometer input 310and pupil/eye tracking step 320. Next, the data from both of these steps310, 320 are processed in data processing step 330. Based on thisprocessing step 330, then the display arms and displays are movedaccording to the data input in step 340. At the same time, the imagegeneration step 350 determines the appropriate type of view that will beprovided to the displays. Finally, the displays are instructed toprovide a particular display in step 360 based on the previous steps310, 320, and 330.

FIG. 4 shows an alternative embodiment of the system 200 includinglenses 15, 16 positioned between the user's eyes 9, 10 and the displays1, 2. All of the components with the same reference numerals in FIG. 4are the same as previously discussed with respect to FIGS. 1A, 1B, and2. In FIG. 4, the lenses 15, 16 are preferably convex and are preferablyarranged on a secondary support 23 connected to the display support arm3. The secondary support 23 is arranged between the user's eyes 9, 10and the displays 1, 2. The convex lenses 15, 16 focus light from thedisplays 1, 2 to make viewing the displays 1, 2 at smaller distancespossible. The head mounted display device of the system 200 places thedisplays 1, 2 at a distance from the user's eyes 9, 10 that is less thana minimum that a human eye can focus. Therefore, the system 200essentially provides very strong reading/magnifying glasses in the formof the convex lenses 15, 16 with, for example, refractive power of 20diopters or more. The convex lenses 15, 16 adjust the images provided bythe displays 1, 2. The convex lenses 15, 16 are physically connected todisplays 1, 2 so that they move with displays, 1, 2 such that the imagesof the displays 1, 2 can be orientated and positioned for the user'seyes 9, 10.

Based on the configuration of the system 100, the displays 1, 2effectively provide an increased field of view while the resolution ofthe image displayed to the user is not reduced. The system 100 alsoallows a more natural and comfortable viewing of the virtual worldbecause the user uses natural movement of the user's eyes as the user'seyes are used to manipulate the virtual world view instead of requiringthe user to move their entire head and neck to change the view of thevirtual world.

Other embodiments can include multiples cameras to track a user's eyes,and may include a myoelectric detector 17, such as shown in FIG. 2. Inanother embodiment, the displays 1, 2 can be stationary, and lightrefracting lenses are moved, instead of the displays 1, 2, as the usermoves their eyes 9, 10. In another embodiment, the two displays 1, 2 arecombined in one display that is moved according to the movements of theuser's eyes 9, 10.

FIG. 6 illustrates a schematic view of the system 100 of FIGS. 1A, 1B,and 2. FIG. 6 provides a simplified representation of the displays 1, 2,the display support arm 3, the motors 4 and 5 arranged in a singlehousing, the processor 7, and the camera 6 facing towards the user'seyes 9, 10.

In another embodiment shown in FIG. 7, one or more projectors 20 areprovided that transmit images to a display surface, genericallyidentified as element 1′. The display surface 1′ can include a surfaceon an interior of a helmet-like headset 11, screen, glass, or othersurface. The projector 20 transmits one or more beams 20′ of images tothe display surface 1′ The beam 20′ can include a set of images, i.e.one image for a left eye and one image for a right eye, that togetherprovide a stereoscopic image for the user's eyes 9, 10. The projector 20is motorized and can change a position of the beam 20′ based on movementof the user's head and/or position of the user's eyes 9, 10. Trackingeye movement, processing the images, and positioning the images displaysbased on a user's eye positioning can be accomplished in a mannersimilar to what has been described.

It will be appreciated that the foregoing is presented by way ofillustration only and not by way of any limitation. It is contemplatedthat various alternatives and modifications may be made to the describedembodiments without departing from the spirit and scope of theinvention. Having thus described the present invention in detail, it isto be appreciated and will be apparent to those skilled in the art thatmany physical changes, only a few of which are exemplified in thedetailed description of the invention, could be made without alteringthe inventive concepts and principles embodied therein. It is also to beappreciated that numerous embodiments incorporating only part of thepreferred embodiment are possible which do not alter, with respect tothose parts, the inventive concepts and principles embodied therein. Thepresent embodiment and optional configurations are therefore to beconsidered in all respects as exemplary and/or illustrative and notrestrictive, the scope of the invention being indicated by the appendedclaims rather than by the foregoing description, and all alternateembodiments and changes to this embodiment which come within the meaningand range of equivalency of said claims are therefore to be embracedtherein.

What is claimed is:
 1. A headset for providing images to a usercomprising: a headset configured to attach to a user's head; displayspositioned adjacent to a user's eyes; a display support arm connected tothe displays; a motor connected to the display support arm that drivesthe display support arm between multiple positions; a camera positionedadjacent to the user's eyes that records the user's eyes; and aprocessor that receives data from the camera regarding a position andmovement of the user's eyes, and the processor uses the data from thecamera to drive the motor and to position the display support arm withrespect to the user's eyes such that the displays maintain a positionadjacent to the user's eyes.
 2. The headset of claim 1, wherein thedisplays include two displays.
 3. The headset of claim 1, wherein themotor includes a pitch motor and a yaw motor.
 4. The headset of claim 1,wherein the camera tracks a user's eye movement.
 5. The headset of claim1, further comprising an accelerometer that measures movement of theuser's head.
 6. The headset of claim 1, further comprising convex lensespositioned between the displays and the user's eyes.
 7. The headset ofclaim 1, further comprising a transmission/receiving unit that sendsdata from the processor to a central processor.
 8. The headset of claim1, wherein the displays project stereoscopic images to the user.
 9. Theheadset of claim 1, wherein the processor generates images related tothe position and the movement of the user's eyes.
 10. A headset forproviding images to a user comprising: a headset configured to attach toa user's head; two displays, each one of the two displays positioned infront a respective one of a user's eyes, such that the two displaysprovide stereoscopic images to the user's eyes; a display support armconnected to the two displays; a pitch motor and a yaw motor, at leastone of the pitch motor or the yaw motor connected to the display supportarm to drive the display support arm between multiple positions; acamera positioned adjacent to the user's eyes that records the user'seyes; an accelerometer that measures movement of a user's head; and aprocessor that receives data from the camera regarding a position andmovement of the user's eyes and data from the accelerometer regardingmovement of the user's head, and the processor uses the data from thecamera and the accelerometer to drive the pitch motor and the yaw motorand to position the display support arm with respect to the user's eyessuch that the two displays maintain a position adjacent to the user'seyes, and images provided to the displays are modified based on the datafrom the camera and the accelerometer.